All using deep learning (artificial neural networks), implemented ourselves mostly using numpy
and sometimes using public frameworks, such as pytorch
and jax
.
If you have no or just a little experience with python, it may be difficult for you to gain the necessary experience fast enough.
Easy to install on your own laptop. The Anaconda distribution is recommended.
Popularity of Python judged by accesses of tutorials on github.
Popularity of Python, and other languages, for Deep Learning
import numpy as np
import matplotlib.pyplot as plt
plt.plot(np.sin(0.1 * np.arange(100)));
How would you do these?
numpy
array) of random values between -0.1 and 0.1# 1. Assign to variable W a 5 x 2 matrix (numpy array) of random values between -0.1 and 0.1
# 2. Assign to variable X a 10 x 5 matrix of random values between 0 and 10.
# 3. Multiply two matrices, X and W.
# 4. Take the transpose of XW.
# 5. Multiply all elements of matrix X by 2 and subtract 1 from all elements.
# Iteratively make small adjustments to an initial matrix `W` to
# make it converge on values in matrix `Z`.
# Iteratively make small adjustments to an initial matrix `W` to
# make it converge on values in matrix `Z`.
Z = np.array([1,2,3])
W = np.random.uniform(-10, 10, 3)
for step in range(300):
W += 0.1 * (Z - W)
if step % 30 == 0:
print(f'{step=} {W=} {Z=}')
Here is a wild statement. In a high-dimensional space, most of the points lie on the surface of the hypercube shell!
Huh? As discussed at this stackexchange post, consider the volume of the hypercube in $d$-dimensional space with each side of the hypercube being of length 1.
$$ 1 \cdot 1 \cdot 1 \cdot \ldots \cdot 1 = 1^d$$Nothing surprising here.
Now, let's look at the volume of the "interior" of this hypercube. Let's define the interior as the space from 0.01 to 0.99 along each dimension. This would be
$$ 0.98 \cdot 0.98 \cdot 0.98 \ldots \cdot 0.98 = 0.98^d$$So what? Well, let's look at the results of these calculations. First let's consider two dimensions.
0.98**2
d = 2
print(f'Total volume {1**d}. Interior volume {0.98**d}')
How about 10 dimensions, or 50?
d = 10
print(f'Total volume {1**d}. Interior volume {0.98**d}')
d = 50
print(f'Total volume {1**d}. Interior volume {0.98**d}')
Okay, this is getting tedious. Let's automate this and calculate interior volume for a range of dimensions up to 100, and plot it.
def interior_volume(d):
return (1 - 0.01 * 2) ** d
dims = np.arange(1, 100, 1)
plt.plot(dims, interior_volume(dims))
plt.xlabel('dimensions')
plt.ylabel('interior volume')
plt.grid('on')
Keep going, up to 1000 dimensions.
dims = np.arange(1, 1000, 1)
plt.plot(dims, interior_volume(dims))
plt.xlabel('dimensions')
plt.ylabel('interior volume')
plt.grid('on')
plt.text(500, 0.6, 'WHOA!', fontsize=40);
Go back to definition of interior_volume
and try a thinner shell.
- Agent Smith from The Matrix
What are the general categories of machine learning?
What will I learn in this class?
Why are we using python and jupyter notebooks, and what are jupyter notebooks?
Will we be using GPUs?
What deep learning software frameworks are there, and which ones will we be using?